{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 文本创作字数控制\n",
    "\n",
    "<span style=\"font-size: 20px; font-weight: bold;\">注意：您使用该案例默认的数据和模型训练时，会产生一定费用。计费方式参考：https://cloud.baidu.com/doc/WENXINWORKSHOP/s/6lrk4bgxb</span>\n",
    "\n",
    "文本创作场景中，大模型可以扮演高效的写作助手角色。大模型具备文本理解、生成及编辑能力，我们可以通过调优场景的Prompt或者通过精调的方式，使得大模型生成精准且符合主题的内容。\n",
    "\n",
    "在利用大模型进行文本创作时，实现字数控制的功能同等重要，尤其在新闻摘要、微博等社交媒体内容创作，都需要精确地控制文本的长度。\n",
    "\n",
    "本文将通过对千帆sdk平台的ERNIE-Speed模型进行微调，优化模型对于文本创作字数控制的功能，并且在实验中验证微调实现的有效性。\n",
    "\n",
    "参考本文的sft,dpo以及sft+dpo实践案例，您也可以更好地上手千帆sdk平台的训练功能。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 0. 环境准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "在此部分，我们将讨论使用千帆平台进行推理任务之前的准备工作。这包括获取访问权限、安装 SDK 等内容。\n",
    "\n",
    "然后通过如下方式设置鉴权所需的 Access Key 和 Secret Key，相关 Key 可以从 [百度智能云控制台 - 安全认证](https://console.bce.baidu.com/iam/#/iam/accesslist) 页面获得。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install 'qianfan>=0.3.16'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "#此处请您根据 SDK 文档获取自己的 access key 和 secret key\n",
    "os.environ[\"QIANFAN_ACCESS_KEY\"] = \"your_qianfan_console_access_key\"\n",
    "os.environ[\"QIANFAN_SECRET_KEY\"] = \"your_qianfan_console_secret_key\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from qianfan.dataset import Dataset\n",
    "from qianfan.trainer.configs import TrainConfig\n",
    "from qianfan.trainer import DPO\n",
    "from qianfan import Completion\n",
    "from qianfan.common import Prompt\n",
    "from qianfan.trainer import LLMFinetune\n",
    "from qianfan.trainer.consts import PeftType\n",
    "from eval import eval\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. 基座模型效果示例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们用本次实验采用的ERNIE-Speed-8K模型进文本创作字数控制的任务。直观感受一下模型微调前的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导入评估集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [06-19 11:50:44] dataset.py:407 [t:8335662592]: no data source was provided, construct\n",
      "[INFO] [06-19 11:50:44] dataset.py:281 [t:8335662592]: construct a qianfan data source from existed id: ds-2hdewmq2w2yw8dz7, with args: {}\n",
      "[INFO] [06-19 11:50:44] dataset.py:480 [t:8335662592]: no destination data source was provided, construct\n",
      "[INFO] [06-19 11:50:44] dataset.py:275 [t:8335662592]: construct a file data source from path: data_file/dpo_test.jsonl, with args: {}\n",
      "[INFO] [06-19 11:50:44] file.py:293 [t:8335662592]: use format type FormatType.Jsonl\n",
      "[INFO] [06-19 11:50:45] utils.py:348 [t:8335662592]: start to get memory_map from /Users/jianruitian/.qianfan_cache/dataset/Users/jianruitian/.qianfan_cache/dataset/.qianfan_download_cache/dg-i3vsgebdzvtec7zb/ds-2hdewmq2w2yw8dz7/1/content/1.arrow\n",
      "[INFO] [06-19 11:50:45] utils.py:276 [t:8335662592]: has got a memory-mapped table\n",
      "[INFO] [06-19 11:50:45] dataset.py:213 [t:8335662592]: change local file format FormatType.Jsonl to qianfan file format FormatType.Jsonl\n"
     ]
    }
   ],
   "source": [
    "ds_test = Dataset.load(qianfan_dataset_id = \"ds-2hdewmq2w2yw8dz7\")\n",
    "ds_test = ds_test.save(data_file=\"data_file/dpo_test.jsonl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们用一个文本归纳总结的任务展示基座模型优化前的效果：\n",
    "* 阅读文本后，提炼出一个30字以内的简洁总结。\n",
    "\n",
    "* 夏日炎炎，一场别开生面的社区环保创意大赛即将拉开帷幕！这不仅仅是一场智慧的较量，更是一次社区凝聚力的提升。让我们一起走进这场充满欢乐与创意的“夏日环保嘉年华——社区篇”。\n",
    "在这个充满活力的季节，我们邀请每个家庭派出2-4名代表，组成充满激情的战队。你们将共同面对一个有趣的挑战：运用生活中的废旧物品或可回收材料，创造出一个个体现夏日风情的环保艺术品。这些作品不仅要体现循环再用的环保精神，还要展现独特创意和社区文化的融合。\n",
    "想象一下，你们的作品或许是一个由废旧塑料瓶打造而成的别致花器，或许是一个用废旧轮胎改造而成的时尚户外座椅，又或许是一盏由废弃玻璃瓶制作而成的别致风灯……无论是什么，它们都将是你们家庭创意的结晶，都将在这个夏天里焕发出新的生机。\n",
    "当然，想要在这场比赛中脱颖而出需要付出不少努力。我们的评委团将根据作品的创新性、实用性和美观性进行细致的评选。所以，请尽情挥洒你们的创意，用双手赋予废旧物品新的生命吧！\n",
    "参与方式也非常便捷。只需通过社区平台在线报名，并上传你们的作品照片。经过评委团的线上初选后，入围的家庭将有机会在线下展示环节大放异彩。最终，我们将评选出冠军1名、亚军2名、季军3名，并为他们颁发精美的证书和丰厚的奖品。\n",
    "这场“夏日环保嘉年华——社区篇”不仅是一次智慧的较量，更是一次社区的欢聚。在这个热情的季节里，让我们共同感受创意的无限可能，体验社区大家庭的温暖，一起度过这个充满欢乐与创意的夏日时光吧！\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [06-19 09:28:42] dataset.py:993 [t:8335662592]: list local dataset data by 5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "社区环保创意大赛邀家庭代表创造环保艺术品，激发创意和凝聚力，线上线下评比颁奖，共庆夏日环保嘉年华。\n"
     ]
    }
   ],
   "source": [
    "p = Prompt(ds_test[5][0]['prompt'])\n",
    "#将任务描述输入到模型进行优化\n",
    "comp = Completion(model=\"ERNIE-Speed-8K\")\n",
    "r = comp.do(prompt=p.render()[0])\n",
    "output = r['result']\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* ERNIE-Speed的回答长度达到了45个词，超出了30字的限制的50%，与预期效果相差甚远。\n",
    "\n",
    "* 该现象属于大模型满足某些特定约束的能力较弱。在此处即字数约束能力不强，出于SFT能够通过对模型输出的概率分布进行精细调整，从而更有效地控制生成文本长度的优势，该问题可以考虑通过准备一些包含明确Prompt和对应Response的语料数据，使用SFT的能力解决。\n",
    "\n",
    "* 因此，我们进行了微调实验，提升生成的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.数据集准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从千帆平台导入sft训练和dpo训练的数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [06-19 11:54:48] dataset.py:407 [t:8335662592]: no data source was provided, construct\n",
      "[INFO] [06-19 11:54:48] dataset.py:281 [t:8335662592]: construct a qianfan data source from existed id: ds-sjv3xchndftmg2fu, with args: {}\n"
     ]
    }
   ],
   "source": [
    "#enable_log(logging.INFO)\n",
    "ds_sft = Dataset.load(qianfan_dataset_id = \"ds-sjv3xchndftmg2fu\")#sft训练集\n",
    "#ds_sft = ds_sft.save(data_file=\"data_file/sft_train.jsonl\")\n",
    "#print(new_ds[0])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [06-19 11:54:50] dataset.py:407 [t:8335662592]: no data source was provided, construct\n",
      "[INFO] [06-19 11:54:50] dataset.py:281 [t:8335662592]: construct a qianfan data source from existed id: ds-ca94jxph35qp1ks3, with args: {}\n"
     ]
    }
   ],
   "source": [
    "ds_dpo = Dataset.load(qianfan_dataset_id = \"ds-ca94jxph35qp1ks3\")#dpo训练集\n",
    "#ds_dpo = ds_dpo.save(data_file=\"data_file/dpo_train.jsonl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们从实际业务场景中得到数据后，需要对样本进行分析和处理。一般包括对原始数据进行清洗、分析数据质量和分布、对数据进行扩充。\n",
    "\n",
    "此处我们的数据已经处理完成，您可以参考[如何使用千帆进行数据处理](https://github.com/baidubce/bce-qianfan-sdk/blob/main/cookbook/dataset/how_to_use_qianfan_operator.ipynb)，来处理您的原始数据数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. 微调训练与测试\n",
    "针对我们的任务，此处设计了六组实验，进行了相应的sft训练，dpo训练以及sft+dpo训练。\n",
    "\n",
    "实验数据如下：\n",
    "| | 实验1 | 实验2 | 实验3 | 实验4(基于实验1)  | 实验5(基于实验2) |\n",
    "|-|-|-|-|-|-|\n",
    "| 精调方法 | sft | sft | dpo | sft+dpo | sft+dpo |\n",
    "| Epoch | 1 | 3 | 1 | 1 | 1 |\n",
    "| Learning Rate | 1e-5 | 1e-5 | 1e-6 | 1e-6 | 1e-6 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 sft精调\n",
    "\n",
    "使用千帆sdk平台的微调功能，此处选择sft训练，设置相应的参数执行训练任务。这里以第二组参数的实验为例，展示平台高效的训练能力。\n",
    "\n",
    "* 以此为例，您可以修改参数，执行其他组别的实验。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 默认参数\n",
    "trainer2 = LLMFinetune(                 #sft训练\n",
    "    train_type=\"ERNIE-Speed\",\n",
    "    name = \"dpo_words_2_fin\",\n",
    "    train_config=TrainConfig(\n",
    "        epoch=3,\n",
    "        learning_rate=1e-5,\n",
    "        #max_seq_len=4096,\n",
    "        peft_type=PeftType.ALL,\n",
    "        #logging_steps=1,\n",
    "        #warmup_ratio=0.10,\n",
    "        #weight_decay=0.0100,\n",
    "        #lora_rank=8,\n",
    "        #lora_all_linear=\"True\",\n",
    "    ),\n",
    "    dataset=ds_sft\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [06-19 09:51:31] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 1% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:52:01] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 3% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:52:32] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:53:02] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:53:33] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:54:03] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:54:34] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:55:04] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:55:35] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:56:05] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:56:36] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 34% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:57:06] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 35% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:57:36] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 37% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:58:07] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 37% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:58:37] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 39% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:59:08] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 40% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 09:59:38] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 40% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:00:09] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 42% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:00:39] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 43% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:01:10] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 44% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:01:40] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 45% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:02:11] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 46% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:02:41] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 47% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:03:12] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 48% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:03:42] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 49% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:04:12] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 50% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:04:12] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:04:43] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 51% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:04:43] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:05:14] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 52% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:05:14] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:05:44] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 53% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:05:44] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:06:15] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 54% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:06:15] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:06:45] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 55% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:06:45] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:07:16] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 56% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:07:16] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:07:46] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 57% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:07:46] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:08:17] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 58% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:08:17] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:08:47] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 60% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:08:47] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:09:18] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 60% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:09:18] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:09:48] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 61% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:09:48] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:10:19] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 63% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:10:19] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:10:49] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 64% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:10:49] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:11:20] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:11:20] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:11:50] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:11:50] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:12:21] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:12:21] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:12:51] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:12:51] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:13:22] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:13:22] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:13:52] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:13:52] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:14:23] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:14:23] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:14:53] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:14:53] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:15:24] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:15:24] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:15:54] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:15:54] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:16:25] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:16:25] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:16:55] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:16:55] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:17:26] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:17:26] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:17:57] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:17:57] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:18:27] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:18:27] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:18:58] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:18:58] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:19:28] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:19:28] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:19:59] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:19:59] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:20:30] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:20:30] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:21:00] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:21:00] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:21:31] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:21:31] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:22:01] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:22:01] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:22:32] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:22:32] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:23:02] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:23:02] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:23:33] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:23:33] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:24:03] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:24:03] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:24:34] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:24:34] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:25:04] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:25:04] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:25:35] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:25:35] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:26:05] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:26:05] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:26:36] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:26:36] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:27:06] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:27:06] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:27:37] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:27:37] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:28:07] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:28:07] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:28:38] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:28:38] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:29:08] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:29:08] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:29:39] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:29:39] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:30:09] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:30:09] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:30:40] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:30:40] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:31:10] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:31:10] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:31:41] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:31:41] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:32:11] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:32:11] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:32:42] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:32:42] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:33:12] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:33:12] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:33:43] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:33:43] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:34:13] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:34:13] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:34:44] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:34:44] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:35:15] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:35:15] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:35:45] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:35:45] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:36:16] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:36:16] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:36:46] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:36:46] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:37:17] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:37:17] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:37:47] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:37:47] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:38:18] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:38:18] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:38:48] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:38:48] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:39:19] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:39:19] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:39:49] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:39:49] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:40:20] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:40:20] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:40:51] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:40:51] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:41:21] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:41:21] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:41:52] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:41:52] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:42:22] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:42:22] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:42:53] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:42:53] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:43:23] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:43:23] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:43:54] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:43:54] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:44:24] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:44:24] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:44:55] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:44:55] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:45:25] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:45:25] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:45:56] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:45:56] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:46:27] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:46:27] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:46:57] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:46:57] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:47:28] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:47:28] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:47:58] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:47:58] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:48:29] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:48:29] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:48:59] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:48:59] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:49:30] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:49:30] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:50:00] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:50:00] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:50:31] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:50:31] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:51:02] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:51:02] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:51:32] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:51:32] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:52:03] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:52:03] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:52:33] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:52:33] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:53:04] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:53:04] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:53:34] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:53:34] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:54:05] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:54:05] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:54:36] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:54:36] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:55:06] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:55:06] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:55:37] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:55:37] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:56:07] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:56:07] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:56:38] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:56:38] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:57:09] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:57:09] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:57:40] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:57:40] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:58:11] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:58:11] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:58:41] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:58:41] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:59:12] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:59:12] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 10:59:43] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 10:59:43] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 11:00:13] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 11:00:13] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 11:00:44] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 65% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 11:00:44] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 11:01:14] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 99% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 11:01:14] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 11:01:45] actions.py:667 [t:10754224128]: [train_action] training ... job_name:dpo_words_2_fin current status: Running, 99% check train task log in https://console.bce.baidu.com/qianfan/train/sft/job-c9qm8jeiy1q7/task-yuc4f8mt3a7y/detail/traininglog\n",
      "[INFO] [06-19 11:01:45] actions.py:674 [t:10754224128]:  check vdl report in https://console.bce.baidu.com/qianfan/visualdl/index?displayToken=eyJydW5JZCI6InJ1bi0xdjV1ZGl0cGZrbjgzaHc5In0=\n",
      "[INFO] [06-19 11:02:15] actions.py:638 [t:10754224128]: [train_action] training task metrics: {'BLEU-4': '20.18%', 'ROUGE-1': '36.33%', 'ROUGE-2': '17.05%', 'ROUGE-L': '31.60%'}\n",
      "[INFO] [06-19 11:02:15] actions.py:639 [t:10754224128]: [train_action] training task checkpoints: []\n",
      "[INFO] [06-19 11:02:15] actions.py:681 [t:10754224128]: [train_action] training job has ended: job-c9qm8jeiy1q7/task-yuc4f8mt3a7y with status: Done\n",
      "[WARNING] [06-19 11:02:15] model.py:95 [t:10754224128]: model id or version_id should be provided\n",
      "[INFO] [06-19 11:02:15] model.py:222 [t:10754224128]: check train job: task-yuc4f8mt3a7y/job-c9qm8jeiy1q7 status before publishing model\n",
      "[INFO] [06-19 11:02:16] model.py:235 [t:10754224128]: model publishing keep polling, current status Done\n",
      "[INFO] [06-19 11:02:18] model.py:273 [t:10754224128]: publishing train task: job-c9qm8jeiy1q7/task-yuc4f8mt3a7y to model: am-z2q56efdehm2/amv-uyv5g0v6xx4e\n",
      "[INFO] [06-19 11:02:49] model.py:298 [t:10754224128]: model am-z2q56efdehm2/amv-uyv5g0v6xx4e published successfully\n",
      "[INFO] [06-19 11:02:49] model.py:278 [t:10754224128]: publish successfully to model: am-z2q56efdehm2/amv-uyv5g0v6xx4e\n"
     ]
    }
   ],
   "source": [
    "trainer2.run()\n",
    "print(trainer2.result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 dpo精调\n",
    "\n",
    "使用千帆sdk平台的微调功能，此处选择dpo训练，设置相应的参数执行训练任务。这里以第二组参数的实验为例，展示平台高效的训练能力。\n",
    "\n",
    "* 以此为例，您可以修改参数，执行其他组别的实验。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer6 = DPO(                 #dpo训练\n",
    "    #train_type=\"ERNIE-Speed-8K\", ###如果没有前置任务，则不需要设置previous_task_id，改在此处设置基座模型\n",
    "    name = \"dpo_words_6\",\n",
    "    previous_task_id = \"task-tdng641si3it\",#第六组实验是基于第二组实验的sft结果进行dpo微调，此处需要设置前序实验的task_id\n",
    "    train_config=TrainConfig(\n",
    "        epoch=1,\n",
    "        learning_rate=1e-6,\n",
    "        max_seq_len=4096,\n",
    "        peft_type=PeftType.ALL,\n",
    "    ),  \n",
    "    dataset=ds_dpo,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer6.run()#执行训练任务"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 实验评估\n",
    "\n",
    "进行完六组实验后，进行评估。此处使用千帆平台的批量推理能力进行评估。\n",
    "\n",
    "评分标准如下：\n",
    "\n",
    "    * 长度控制得分：针对不同参数设置的输出进行字符数统计，定义以下长度控制得分规则，对不同参数设置的训练方式求一个平均分：\n",
    "    \n",
    "    首先计算预测字符数与输入字符限制的比例减一的绝对值，明确两者之间的差距，然后根据这个绝对值的大小返回不同的得分。\n",
    "    如果这个绝对值在0到0.05之间（也就是说，预测字符数与输入字符限制的比例在1±0.05之间），则得分为1；随着这个绝对值的增大，得分逐渐降低，当绝对值超过0.25时，得分为0。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导入评估集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [06-18 23:42:30] dataset.py:407 [t:8335662592]: no data source was provided, construct\n",
      "[INFO] [06-18 23:42:30] dataset.py:281 [t:8335662592]: construct a qianfan data source from existed id: ds-2hdewmq2w2yw8dz7, with args: {}\n"
     ]
    }
   ],
   "source": [
    "dpo_test = Dataset.load(qianfan_dataset_id = \"ds-2hdewmq2w2yw8dz7\")#dpo评估集\n",
    "# dpo_test = dpo_test.save(data_file=\"data_file/dpo_test.jsonl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此处，我们以第六组实验的训练结果为例，展示批量离线推理和评估的过程。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5899999999999999\n"
     ]
    }
   ],
   "source": [
    "#第一个参数填写对应实验的模型版本id，第二个参数填写用于评估的数据集id\n",
    "res = eval(\"amv-wsr34aidpqcr\", dpo_test)\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最终的评估结果如下：\n",
    "| | 实验1 | 实验2 | 实验3 | 实验4(基于实验1)  | 实验5(基于实验2) |\n",
    "|-|-|-|-|-|-|\n",
    "| 精调方法 | sft | sft | dpo | sft+dpo | sft+dpo |\n",
    "| Epoch | 1 | 3 | 1 | 1 | 1 |\n",
    "| Learning Rate | 1e-5 | 1e-5 | 1e-6 | 1e-6 | 1e-6 |\n",
    "| 长度得分 | 0.629 | 0.468 | 0.733 | 0.744 | 0.590|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，对于文本的字数控制任务，dpo的表现远远优于sft。\n",
    "\n",
    "单独使用dpo训练以及在sft基础上进行dpo训练的效果都好于单独sft训练的效果。\n",
    "\n",
    "您可以参考本文的方法，修改训练方法，尝试训练您自己的模型。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
